Automated patient modeling and positioning

ABSTRACT

Automated patient positioning and modelling includes a hardware processor to obtain image data from an imaging sensor, classify the image data, using a first machine learning model, as a patient pose based on one or more pre-defined protocols for patient positioning, provide a confidence score based on the classification of the image data and if the confidence score is less than a pre-determined value, re-classify the image data using a second machine learning model; or if the confidence score is greater than a pre-determined value, identify the image data as corresponding to a patient pose based on one or more pre-defined protocols for patient positioning during a scan procedure.

FIELD

The aspects of the disclosed embodiments relate generally to patientpositioning systems for medical scanning, and more particularly to asoft pose classifier for automated patient positioning and modelling.

BACKGROUND

In medical scanning procedures, such as computed tomography (CT) orMagnetic Resonance Imaging (MRI), a sensor-aided computation algorithmcan help automatically position the patient for scanning. Thesealgorithms are generally implemented in order to assist with the properpositioning of the patient for the medical scanning procedure, alsoreferred to herein as scan or scanning.

In current automated patient positioning workflows for medical scanning,a hard or rigid classification methodology is adopted for identifying apatient pose. These rigid classification systems will set a hardthreshold for prediction and generally output only one inference resultamong certain pre-defined options. These options might merely identifythat the patient's head is towards the gantry, that the feet are towardsthe gantry, or that the patient is in a supine, prone or lateralposition. There is no classification information or inference resultthat provides a more detailed analysis of the patient pose.

A rigid classification system is typically in the form of a trained deepneural network. Given the performance limitations of deep neuralnetworks, the classification performance can be easily compromised byinterference or disturbance signals in the collected sensor data orenvironmental changes. This can produce incorrect positioningpredictions. It would be advantageous to be able to provide a soft poseclassification model which considers predicted probabilities for allhigh-confidence classes.

Accordingly, it would be desirable to provide methods and apparatus thataddress at least some of the problems described above.

SUMMARY

The aspects of the disclosed embodiments are directed to a method,apparatus and system for automated patient positioning and modelling.This and other advantages of the disclosed embodiments are providedsubstantially as shown in, and/or described in connection with at leastone of the figures, as set forth in the independent claims. Furtheradvantageous modifications can be found in the dependent claims.

According to a first aspect, the disclosed embodiments provide a methodfor automated patient positioning and modelling. In one embodiment, themethod includes obtaining, by a hardware processor, image data from asensor. The obtained image data is classified as a patient pose using afirst machine learning model. A confidence score of the patient pose isprovided based on the classification of the image data. If theconfidence score is less than a pre-determined value, the image data isre-classified using a second machine learning model. If the confidencescore is greater than a pre-determined value, the image data isidentified as a patient pose corresponding to one or more predefinedprotocols for patient positioning. The aspects of the disclosedembodiments are configured to determine whether the patient pose on thegantry, for example, is consistent with one or more pre-definedprotocols for a scanning procedure.

In a possible implementation form, the first machine learning model canbe or is an ensemble model and the second machine learning model can beor is a deep convolutional neural network model.

In a possible implementation form, the image data is patient pose imagedata.

According to a second aspect the disclosed embodiments provide a systemfor automated patient positioning and modelling. In one embodiment, thesystem includes one or more imaging sensors and a hardware processor.The hardware processor is configured to receive image data from the oneor more imaging sensors. The hardware processor is further configured toclassify the image data, using a first machine learning model, as apatient pose based on one or more pre-defined protocols for patientpositioning. The hardware processor will provide a confidence scorebased on the classification of the image data. If the confidence scoreis less than a pre-determined value, the hardware processor willre-classify the image data using a second machine learning model. If theconfidence score is greater than a pre-determined value, the hardwareprocessor will identify the image data as corresponding to a correctpatient pose based on the one or more pre-defined protocols for patientpositioning.

According to a third aspect the disclosed embodiments are directed to acomputer program product. In one embodiment, the computer programproduct has a non-transitory computer-readable medium withmachine-readable instructions stored thereon. The execution of themachine-readable instructions by a computer will cause the computer toobtain image data from a sensor and classify the image data using afirst machine learning model as a patient pose. A confidence score isprovided based on the classification of the image data. If theconfidence score is less than a pre-determined value, the image data isre-classified using a second machine learning model. If the confidencescore is greater than a pre-determined value, the image data isidentified as a patient pose corresponding to one of a correct patientpose.

These and other aspects, implementation forms, and advantages of theexemplary embodiments will become apparent from the embodimentsdescribed herein considered in conjunction with the accompanyingdrawings. It is to be understood, however, that the description anddrawings are designed solely for purposes of illustration and not as adefinition of the limits of the disclosed invention, for which referenceshould be made to the appended claims. Additional aspects and advantagesof the invention will be set forth in the description that follows, andin part will be obvious from the description, or may be learned bypractice of the invention. Moreover, the aspects and advantages of theinvention may be realized and obtained by means of the instrumentalitiesand combinations particularly pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following detailed portion of the present disclosure, theinvention will be explained in more detail with reference to the exampleembodiments shown in the drawings, in which:

FIG. 1 is a network environment diagram of an exemplary system forautomated patient positioning and modelling in accordance with theaspects of the disclosed embodiments.

FIG. 2 is a schematic illustration of an exemplary scenario forimplementation of a system for automated patient positioning andmodelling incorporating aspects of the disclosed embodiments.

FIG. 3 is a process flow diagram illustrating aspects of the automatedpatient positioning and modelling of the disclosed embodiments.

FIG. 4 is a flowchart illustrating a method incorporating aspects of thedisclosed embodiments.

FIG. 5 is a block diagram of exemplary components of a serverarchitecture for automated patient positioning and modelling inaccordance with the aspects of the disclosed embodiments.

DETAILED DESCRIPTION OF THE DISCLOSED EMBODIMENTS

The following detailed description illustrates exemplary aspects of thedisclosed embodiments and ways in which they can be implemented.Although some modes of carrying out the aspects of the disclosedembodiments have been disclosed, those skilled in the art wouldrecognize that other embodiments for carrying out or practising theaspects of the disclosed embodiments are also possible.

Referring to FIG. 1 , a schematic block diagram of an exemplary system100 for automated patient position and modelling is illustrated. Theaspects of the disclosed embodiments are generally directed toautomatically identifying patient poses prior to an imaging or scanningprocedure and determining whether the patient pose is aligned with therequired scanning protocols, referred to herein as pre-definedprotocols. The soft pose classification system of the disclosedembodiments is intended to be implemented in different medical imagingenvironments, including, but not limited to CT, X-Ray and MRI.

As shown in FIG. 1 , the system 100 generally includes a computingdevice or server 102. Although a server is generally referred to herein,the aspects of the disclosed embodiments are not so limited. Inalternate embodiments, the server 102 can include any suitable computeror computing arrangement.

In one embodiment, the server 102 includes a processor 104, such as ahardware processor. Although only one processor 104 is generallydescribed herein, the aspects of the disclosed embodiments are not solimited. In alternate embodiments, the server 102 can include anysuitable number of processors 104.

The system 100 is generally configured to take an image as the inputdata, which is obtained from one or more imaging or optical sensors 112.In one embodiment, the hardware processor 104, either alone or incombination with other components of the system 100, is generallyconfigured to obtain the input data, the images or image data, from oneor imaging sensor(s) 112 disposed in an imaging room 110. The processor104 is configured to classify the image data, using a first machinelearning model, also referred to as pose inference module 106, as apatient pose. The label for the patient pose will generally be based onone or more pre-defined protocols for patient positioning during or inconjunction with a scanning or imaging procedure.

A confidence score is then provided by the pose inference model 106based on the classification of the image data by the pose inferencemodel 106. The confidence score reflects the assessment by the poseinference model 106 that the prediction corresponds to the indicatedpose position label. Examples of such pose position labels, include, butare not limited to “supine feet toward gantry”, “prone feet towardgantry” or “prone head toward gantry.” These are merely examples and arenot intended to limit the scope of the claimed subject matter.

If the confidence score is less than a pre-determined value, meaningthat the pose inference model 106 cannot reliably determine or identifythe pose from the image data, the hardware processor 104 is configuredto re-classify the image data using a second machine learning model. Inone embodiment, the second machine learning model can be or is a deepconvolutional neural network model.

If the confidence score is greater than a pre-determined value, meaningthat the pose inference model 106 has determined that the image datacorresponds to a certain pose, the image data is identified or otherwiselabelled as corresponding to a patient pose based on one or morepre-defined protocols for patient positioning.

In one embodiment, the apparatus or system 100 can also include a userinterface 130. The user interface 130 can be communicatively coupled tothe server 102 and is configured to provide an output of the posepredictions, as is generally described herein. For example, in oneembodiment, the user interface 130 can provide a list of the top “n”predictions of the pose inference model 106.

Generally, the automated patient positioning and modelling will beimplemented and carried out in an imaging room 110, such as in ahospital or medical facility. In one embodiment, the imaging area orroom 110 is a medical scanning or imaging room where medical imaging isperformed. The aspects of the disclosed embodiments can be implementedin any suitable medical imaging environment, including, but not limitedto, CT, MRI and X-Ray, for example.

The imaging room 110 will generally include at least one image capturedevice 112, and an imaging platform 114, also referred to as a gantry.The image capture device 112, which might also be referred to as acamera, will generally comprise an image or optical sensor. Examples ofsuitable sensors include, but are not limited to, red-blue-greensensors, depth sensors, a digital camera, an image sensor, a nightvision capable camera, a video recorder, a CCTV camera, and other typesof image-capture devices. In alternate embodiments, any suitable imagesensor or device can be used to capture patient pose information.

The image capture device(s) 112 are disposed or installed at specificlocation(s) within the imaging area 110 to adequately capture images ofa patient, also referred to herein as poses, on the imaging platform114. In one embodiment, the image capture device 112 is installed on orin connection with a ceiling of the imaging room 110. In this manner,the imaging platform 114, and the patient that is disposed on or inconnection with the imaging platform 114, is within a field of view 116of the image capture device 112. Although the description here isgenerally with respect to the ceiling of the imaging room, the aspectsof the disclosed embodiments are not so limited. In alternateembodiments, the image sensor 112 or camera can be located at or inconnection with any suitable location in the room, including forexample, the sidewalls or on the imaging device itself. The aspects ofthe disclosed embodiments are generally directed to providing areadiness check of the patient positioning before or as the patient issent into the gantry.

FIG. 2 illustrates an exemplary implementation of a system 100incorporating aspects of the disclosed embodiments. In this example, apatient platform 114 is disposed in an imaging room 110. An imagesensor(s) 112 is disposed in the room 110, which in this example isabove the patient platform 114, such as on the ceiling. The image sensor112 is disposed so that the patient platform 114 is disposed within thefield of view 116 of the image sensor 112. An exemplary gantry 122 isdisposed in conjunction with patient platform 114. The server 102 iscommunicably coupled to the image sensor 112 and is configured toreceive the captured image data as an input.

In the example of FIG. 2 , the image sensor 112 is disposed on theceiling of the imaging room 110. The field of view 116 of the imagecapture device 112 will generally encompass a wide area, including forexample, the scanner platform or gantry 114. Generally, the patientmight be lying down during the image capture process. Alternatively, thepatient could be sitting or in some other position for the image captureprocess. The aspects of the disclosed embodiments are not intended to belimited by the particular position of the patient during the imagecapture process.

For each scanning case, a patient first arrives at the medical scanningroom 110 and enters into the field of view 116 of the image sensor(s)112. The patient is then positioned on the patient platform or scannerbed 114 and prepares for the scanning. This positioning can beimplemented in any suitable manner. The patient can lie down or sit, ortake any suitable position needed for the scanning. In one embodiment,an off-the-shelf person detection algorithm can be applied to determineif the patient is roughly positioned in the field of view 116 of thesensor 112.

In one embodiment, once it is detected that the patient is in a suitableposition with respect to the imaging platform 114, such as for example,lying down, the image sensor(s) 112 will capture images, or posepositions, of the patient. Generally, the images will be captured withrespect to gantry 114. This can include for example, capturing images ofpre-defined positions that correspond to joint locations of the patient.In one embodiment, the sensor or image data can be transmitted to theprocessor 104 and the pose inference model 106 for pose inference, as isdescribed herein.

Instead of rigid classification, which outputs only one inference resultamong certain pre-defined options, the pose inference model 106 of thedisclosed embodiments is configured to consider all possible positionswhich achieve high probability predictions. The pose inference model 106of the disclosed embodiments will then output the top “n” predictionsalong with a computed confidence score, based on classifier predictionto the user.

For example, a first pose image, pose 1, (supine feet toward gantry)receives 0.4 as confidence score. A second pose image, pose 2 (pronefeet toward gantry) receives 0.9 as a confidence score. The nth poseimage, pose n (prone head toward gantry) receives a 0.004 as theconfidence score. As will generally be understood, any suitable numberof pose images can be captured.

In one embodiment, the apparatus 100 is configured to provide a list ofthe confidence scores to the user. The list can be presented, forexample, on or via the user interface 130 of the apparatus 100.

In one embodiment, the list of confidence scores above can be rankedwith the top scores appear first. For the example above, the scores arelisted in the following order: 1) pose 2 (prone feet toward gantry), 2)pose 1 (supine feet toward gantry), . . . , n) pose n (prone head towardgantry). In this manner, the user, such as a technician, can choose orotherwise confirm any one of these options by also looking at the realpatient pose.

The user in this example, typically a technician, can use the list ofthe top “n” predictions to either confirm the patient positioning priorto the imaging process, or re-position the patient. In one embodiment,the option list contains the correct patient pose so that the techniciancan manually choose the correct pose. The real or actual patient posemay not be aligned with the pose required for the pre-defined protocols.

The system 100 is configured to check if the chosen pose is aligned withpre-defined protocols which require a certain pose for medical scanningand provide instructions accordingly. The aspects of the disclosedembodiments provide a more user friendly positioning system that waspreviously realized.

In one embodiment, if the prediction output of the pose inference model106 does not align with the predefined protocol, rather than just awarning message, the system 100 is configured to output one or more highconfidence predictions based on a soft threshold. The confidenceprediction(s) can be presented to the technician on a suitable userinterface. The technician can choose from the one or more confidencepredictions. After the technician makes the choice, the system 100 willfirst compare the current pose of the patient as confirmed by the userand the pre-defined scanning protocols. The system 100 will then provideinstructions for the patient to adjust his/her pose. The instructions,which can be provided verbally or displayed on a screen installed in themedical scanning room are based on the difference(s) between the currentpatient pose and the predefined patient poses. This can avoidcomplicated and redundant operation, such as confirming the warningmessage and then manually making an adjustment to correct the patientpose.

The aspects of the disclosed embodiments will provide possible optionsfor user to choose from. For example, the system 100 is configured topresent several options of the possible patient poses with decreasingpossibilities, such as for example, 1) supine feet towards gantry or 2)lateral feet towards gantry. The user will be asked to choose from oneof these options. The system 100 then checks whether the user choice,which should reflect the actual patient pose and the pose required bythe predefined protocol, align. If there is alignment between the userchoice and the pose of the predefined protocol, the system 100 caninitiate the scanning process. If the user choice does not align withthe pose required by the predefined protocol, the system 100 isconfigured to inform one or both of the technician and patient. In thismanner, further pose adjustment can take place until the system 100confirms that the patient pose is correct for the scanning.

The aspects of the disclosed embodiments facilitate online modellearning for more accurate model performance. For example, if thetechnician chooses a pose that is different from the model prediction,this means that the original prediction from the pose inference model106 is not correct. The pose inference model 106 can take new data, suchas the 2D keypoint data, sensor data collected from image capture device112, the pose label from the technician, and the original wrongprediction, for further online learning/finetuning of the pose inferencemodel 106, such as by reinforcement learning.

In the typical patient positioning system with hard pose identification,the system will show warning messages if the predicted patient pose doesnot align with pre-defined scanning protocol. The technician then needsto manually confirm or bypass the warning message, and manually correctthe model prediction if it is incorrect. In soft pose identificationprocess of the disclosed embodiment, this situation will be alleviated,even if the model 106 is not confident about the prediction.

Rather than providing warning messages, the aspects of the disclosedembodiments present options to the user, generally through the userinterface 130. In this manner, the aspects of the disclosed embodimentsallow the user to directly choose the correct pose when the model 106 isnot confident about its own prediction. This can be more efficient anduser-friendly as compared to hard pose cases, which only issue warnings.

In one embodiment, the aspects of the disclosed embodiments enable theuser(s) to input the ground-truth pose information with one click. Forexample, as also described above, the provided option list will includethe correct patient pose. The technician can manually choose the correctthat pose option that reflects the real patient pose by “clicking” theselection on the user interface 130. The system 100 is configured tocheck if the selected pose is aligned with the pose required for thepre-defined protocols. The system 100, and in particular the poseinference model 106, will then be corrected/updated given the userinput. By incorporating user-in-the-loop fashion in the system 100, thepose inference model 106 of the disclosed embodiments can be updated andperform increasingly better.

FIG. 3 illustrates an exemplary flow diagram of a workflow 300incorporating aspects of the disclosed embodiments. In this example,sensor data from the image sensor 112 of FIG. 1 , is acquired 302 fromone or both of Modality A and Modality B, where modality refers todifferent types of imaging sensors 112. Examples of the differentmodalities can include, but are not limited to, digital cameras, cameramodules, camera phones, optical mouse devices, medical imagingequipment, night vision equipment such as thermal imaging devices, andothers.

In one embodiment, a two dimensional key point detection (2D Key pointDetection) 306 is used to verify the image data as joint locationimages. Generally, 2D Key point Detection is used to acquire images ofpre-determined joint locations. For example, an image of the patient inthe field of view 116 of FIG. 2 is captured by a camera. The image isthen processed using a 2D Key point Detection algorithm to detect and/oridentify pre-determined joints or joint locations.

The output of the 2D Key point Detection 306, the 2D joint locations ofthe predefined body joints, is provided to a first machine learningalgorithm 310. In one embodiment, the first machine learning algorithm310 is an ensemble model. In alternate embodiments, the first machinelearning algorithm 310 can comprise any suitable machine learning modelthat 2D body keypoint locations as inputs and outputs classificationprobability predictions. The training typically optimizes modelparameters given certain amount of training data. In one embodiment, thetraining data includes pairs of 2D keypoint joint locations and thecorresponding pose class labels.

The ensemble model 310 is configured to generate a classification orconfidence score 314 that is associated with the sensor image data. Inone embodiment, the confidence score 314 is normalized from 0 to 1,where a score of 0 is associated with a low confidence and a score of 1is associate with a high confidence. A low confidence score is assignedwhen the model 310 is not confident that the patient is in the correctposition or pose for the imaging. A low confidence score does not meanthat the pose of the patient is incorrect. Rather, the low confidencescore can imply that the model 310 was not able to determine that thepose is correct.

A high confidence score is assigned 316 when the model 310 is confidentthat the patient is in the correct position for the particular imagingprocess. In other words, a high confidence score is assigned when themodel 310 is confident about its own prediction. While, in the unlikelysituation that the model prediction may is not correct, even with a highconfidence score, system 100 is configured to allow the technician tomanually intervene and correct the prediction of the model 310.

If the confidence score indicates a high confidence 316 in theprediction, a pose estimation output 320 is provided. The output 320will be a class label indicating which pose the patient is currently in.An example of such a label is “supine with feet towards gantry.” Thesystem 100 is configured to check whether this label aligned with theselected protocol. For example, chest CT requires the patient to be inthe “supine feet towards gantry pose.” If the output 320 in this exampleis the label “supine feet towards gantry pose”, the output 320 of poseestimation confirms the patient pose is correct and then the system 100will start the corresponding scanning process.

If the confidence score indicates a low confidence 318, meaning that themodel 310 is not confident the patient is in a certain pose, the inputdata 302 is processed by a second machine learning model 312. In thisexample, the second machine learning model is a more detailed andextensive machine learning model. One example of such a model is a deepconvolutional neural network. The second machine learning model willgenerally have been trained on a much larger training set than the firstmachine learning model. The second machine learning model will also bemore time and resource intensive, requiring more inference and memory.The output of the second machine learning module 312 will be the poseestimation output 320. In one embodiment, the output 320 can provide thepossible patient pose predictions in decreasing possibilities, such as“1) supine feet towards gantry” or “2) lateral feet towards gantry.” Theuser can then choose from one of these options to initiate thecorresponding scanning process.

In one aspect, the disclosed embodiments include a training phase and anoperational phase. In the training phase, the pose model 106 is trained,using training data, to enable the pose model 106 to perform specificintended functions in the operational phase. The processor 104 isconfigured to execute supervised training of the pose model 106 usingtraining data of images of the pre-defined pose or joint positions toobtain a trained pose model 106. The training data contains pairs ofsensor data with labeled pose annotation, or pairs of 2D keypointlocations with labeled pose annotation.

As an example, during the training of the pose model 106, images, forexample normal 2D images, of pre-defined joint positions and some imageswith abnormality are fed to the pose model 106. Initially, the posemodel 106 is not provided with information about where the abnormalityis present in the images having abnormality. The pose model 106 isconfigured to automatically finds the abnormality in the images. Theterm “abnormality” refers to deviation in following the defined set ofprotocols and procedures while performing a specific task. For example,in an image a joint may be detected at a location which may be notcorrespond to its designated location.

In accordance with an embodiment, the training data includes images, ora sequence of images, provided by image-capture devices to the server102. Optionally, the training data of images of the pre-defined jointlocations is pre-stored in the server 102. Based on the training of thepose model 106, a trained pose model is obtained which is used in theoperational stage of the system 100.

In operation, the processor 104 is configured to obtain the poseimage(s) from the image-capture device(s) 112. The processor 104receives the image(s) via the communication network 108. In oneembodiment, the image(s) is obtained in real time or near-real time assuch images are captured. Optionally, an image has a time stampassociated therewith. In an embodiment, the server 102 is configured tostore information about a location associated with each of the pluralityof image-capture devices 112.

In one embodiment, the processor 104 is configured to communicate analert together with visual information, to a specified electronicdevice, such as a smartphone or other portable device, that is mapped orotherwise communicatively coupled to a user, such as a technician. Thealert is communicated based on the pose estimation output 320. Thevisual information is a visual explanation indicative of a reason of thealert. The alert together with visual information is communicated as apart of the action associated the pose estimation output.

As an example, in a hospital environment, the processor 104 communicatesthe alert together with visual information to a smartphone of a hospitalpersonnel, such as a doctor or a lab assistant. The hospital personnelmay be associated with a specific location, such as an MRI room in casethe abnormality is detected in the MRI room.

The electronic device may include, but is not limited to a cellularphone, personal digital assistants, handheld devices, wireless modems,laptop computers, personal computers and the like. The electronicdevices can be mapped with registered users and communicatively coupledto the processor 104.

In another implementation, the processor 104 is further configured tocommunicate an instruction to a suitably configured medical deviceequipment or technician. In one embodiment, the instruction iscommunicated based on the pose estimation output.

FIG. 4 illustrates one embodiment of a method 400 incorporating aspectsof the disclosed embodiments. In this example, an image(s) correspondingto a pose is taken as the input data 402. As described above, the image,or images, are captured by suitable image capture devices. The poseimage input data is then classified 404 using a machine learning model.The machine learning model of the disclosed embodiments is configured toautomatically identify patient poses from the input image data anddetermine if the input image data is aligned with required scanningprotocols.

It is determined 406 whether a confidence score associated with theclassification is a high score or a low score. A high score is generallyassociate with a high probability prediction that the input image datais aligned with the required scanning protocol. A low score indicates alow probability prediction that the input image data is aligned with therequired scanning protocol.

If the confidence score is a high score, a pose estimate output isprovided 408. The pose estimation output provides the top “n”predictions along with the confidence score based on the classifierprediction. The user, such as a technician, can make a choice from theseveral high confidence predictions and initiate 410 the scan.

If the confidence score is a low score, the pose inference model of thedisclosed embodiments cannot reliably determine or identify the posefrom the input image data. In this case, the input image data isreclassified 412 using a second machine learning model. The secondmachine learning model is generally a more intensive and complicatedclassification model, such as a deep convolutional neural network model.

Referring again to FIG. 1 , the server 102 generally includes suitablelogic, circuitry, interfaces and/or code that is configured to receiveone or more pose images from the image-capture device(s) 112 and processthose images as is generally described herein. In some embodiments, theserver 102 can be configured to receive a sequence of image frames (e.g.one or more video) of the patient from the image-capture device(s) 112.Examples of the server 102 may include, but are not limited to, anapplication server, a web server, a database server, a file server, acloud server, or a combination thereof.

The processor 104 generally includes suitable logic, circuitry,interfaces and/or code that is configured to process the image(s) (orthe sequence of image frames) as is generally described herein. In oneembodiment, this can also include the pose module 106. The processor 104is configured to respond to and process instructions that drive thesystem 100. Examples of the processor 104 include, but are not limitedto, a microprocessor, a microcontroller, a complex instruction setcomputing (CISC) microprocessor, a reduced instruction set (RISC)microprocessor, a very long instruction word (VLIW) microprocessor, orany other type of processing circuit. Optionally, the processor 104 maybe one or more individual processors, processing devices and variouselements associated with a processing device that may be shared by otherprocessing devices. Additionally, the one or more individual processors,processing devices and elements are arranged in various architecturesfor responding to and processing the instructions that drive the system100. Although a server and hardware processor are generally describedherein, the aspects of the disclosed embodiments are not so limited. Inalternate embodiments, the system 100 can include any suitablecomponents or devices that are needed to carry out the processesdescribed herein, such as a memory or storage, for example.

In one embodiment, the patient pose model 106 can comprise or be part ofa standalone computing device, in communication with, or part of, thehardware processor 104. In one embodiment, the patient pose model 106will include or be connected to the machine learning models needed tocarry out the aspects of the disclosed embodiments described herein.

In one embodiment, the server 102 is communicatively coupled to theimage capture device(s) 112 via the communication network 108. Thecommunication network 108 includes a medium through which theimage-capture device(s) 112 and the server 102 communicate with eachother. The communication network 108 may be a wired or wirelesscommunication network. Examples of the communication network 108 mayinclude, but are not limited to, a Wireless Fidelity (Wi-Fi) network, aLocal Area Network (LAN), a wireless personal area network (WPAN), aWireless Local Area Network (WLAN), a wireless wide area network (WWAN),a cloud network, a Long Term Evolution (LTE) network, a plain oldtelephone service (POTS), a Metropolitan Area Network (MAN), and/or theInternet. The image-capture device(s) 112 are configured to connect tothe communication network 108, in accordance with various wired andwireless communication protocols. Examples of such wired and wirelesscommunication protocols may include, but are not limited to,Transmission Control Protocol and Internet Protocol (TCP/IP), UserDatagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), FileTransfer Protocol (FTP), ZigBee, EDGE, infrared (IR), IEEE 802.11,802.16, Long Term Evolution (LTE), Light Fidelity (Li-Fi), and/or othercellular communication protocols or Bluetooth (BT) communicationprotocols, including variants thereof. Optionally or in addition to, oneor more medical equipment or device(s) 118 and/or medical imagingdevice(s) 120 are communicatively coupled to the server 102.

FIG. 5 is a block diagram 500 of exemplary components of a server forautomated patient positioning in accordance with the aspects of thedisclosed embodiments. FIG. 2 is described in conjunction with elementsfrom FIG. 1 . With reference to FIG. 1 , there is shown the server 102.In this example, the server 102 includes a memory 402, a networkinterface 504, the processor 104, and the pose classification model 106.The processor 104 is communicatively coupled to the memory 402, thenetwork interface 504, and the pose classification model 106.

The memory 502 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to store instructions executable bythe processor 104. The memory 502 is further configured to store theimage(s) from the image sensor 112. The memory 502 may be furtherconfigured to store operating systems and associated applications of theserver 102 including the pose classification model 106. Examples ofimplementation of the memory 502 may include, but are not limited to,Random Access Memory (RAM), Read Only Memory (ROM), Hard Disk Drive(HDD), Flash memory, and/or a Secure Digital (SD) card. A computerreadable storage medium for providing a non-transient memory mayinclude, but is not limited to, an electronic storage device, a magneticstorage device, an optical storage device, an electromagnetic storagedevice, a semiconductor storage device, or any suitable combination ofthe foregoing.

The network interface 504 includes suitable logic, circuitry, and/orinterfaces that is configured to communicate with one or more externaldevices, such as the image-capture device(s) 112, the medical equipment118, 120 or an electronic device (such as a smartphone) shown in FIG. 1. Examples of the network interface 504 may include, but is not limitedto, a radio frequency (RF) transceiver, an antenna, a telematics unit,one or more amplifiers, one or more oscillators, a digital signalprocessor, a coder-decoder (CODEC) chipset, and/or a subscriber identitymodule (SIM) card. Optionally, the network interface 404 may communicateby use of various wired or wireless communication protocols.

The aspects of the disclosed embodiments are directed to a soft poseclassification model based on deep/machine learning algorithms that canbe directly integrated into any existing patient positioning systemworkflow. The model, along with its associated system, is able toautomatically identify or predict patient poses and check if they arealigned with requiring scanning protocols. The system of the disclosedembodiments outperforms existing state-of-the-art systems and is robustto adversarial attacks, which is a well-known potential risk for most ofexisting deep learning-based models and systems.

Various embodiments and variants disclosed above, with respect to theaforementioned system 100, apply mutatis mutandis to the method. Themethod described herein is computationally efficient and does not causeprocessing burden on the processor 104.

Modifications to embodiments of the aspects of the disclosed embodimentsdescribed in the foregoing are possible without departing from the scopeof the aspects of the disclosed embodiments as defined by theaccompanying claims. Expressions such as “including”, “comprising”,“incorporating”, “have”, “is” used to describe and claim the aspects ofthe disclosed embodiments are intended to be construed in anon-exclusive manner, namely allowing for items, components or elementsnot explicitly described also to be present. Reference to the singularis also to be construed to relate to the plural.

Thus, while there have been shown, described and pointed out,fundamental novel features of the invention as applied to the exemplaryembodiments thereof, it will be understood that various omissions,substitutions and changes in the form and details of devices and methodsillustrated, and in their operation, may be made by those skilled in theart without departing from the spirit and scope of the presentlydisclosed invention. Further, it is expressly intended that allcombinations of those elements, which perform substantially the samefunction in substantially the same way to achieve the same results, arewithin the scope of the invention. Moreover, it should be recognizedthat structures and/or elements shown and/or described in connectionwith any disclosed form or embodiment of the invention may beincorporated in any other disclosed or described or suggested form orembodiment as a general matter of design choice. It is the intention,therefore, to be limited only as indicated by the scope of the claimsappended hereto.

What is claimed is:
 1. A method for automated patient positioning andmodelling, the method comprising using a hardware processor to: obtainimage data from an imaging sensor; classify the image data, using afirst machine learning model, as a patient pose based on one or morepre-defined protocols for patient positioning; provide a confidencescore based on the classification of the image data; and if theconfidence score is less than a pre-determined value, re-classify theimage data using a second machine learning model; or if the confidencescore is greater than a pre-determined value, identify the image data ascorresponding to a patient pose based on the one or more pre-definedprotocols for patient positioning.
 2. The method according to claim 1further comprising initiating a scan corresponding to the one or morepre-defined protocols for patient positioning when the image data isidentified as corresponding to the patient pose based on the one or morepre-defined protocols for patient positioning.
 3. The method accordingto claim 1 wherein the first machine learning model is an ensemble modeland the second machine learning model is a deep convolutional neuralnetwork model.
 4. The method according to claim 1, wherein the imagedata is patient pose image data.
 5. The method according to claim 1,further comprising providing a ranking of confidence scores and enablinga selection of a pose associated with a confidence score.
 6. The methodaccording to claim 5, further comprising comparing the selected pose toa required pose of the pre-defined protocol; and confirming the selectedpose as corresponding to the pose of the pre-defined protocol andbeginning the scan; or provide pose adjustment instructions if theselected pose is not confirmed.
 7. A system for automated patientpositions and modelling, comprising: an imaging sensor; and a hardwareprocessor configured to receive image data from the imaging sensor,wherein the hardware processor is further configured to: classify theimage data, using a first machine learning model, as a patient posebased on one or more pre-defined protocols for patient positioning;provide a confidence score based on the classification of the imagedata; and if the confidence score is less than a pre-determined value,re-classifying the image data using a second machine learning model; orif the confidence score is greater than a pre-determined value, identifythe image data as corresponding to a patient pose based on the one ormore pre-defined protocols for patient positioning.
 8. The systemaccording to claim 7, wherein the hardware processor is furtherconfigured to initiate a scan corresponding to the one or morepre-defined protocols for patient positioning when the image data isidentified as corresponding to the patient pose based on the one or morepre-defined protocols for patient positioning.
 9. The system accordingto claim 7, wherein the first machine learning model is an ensemblemodel and the second machine learning model is a deep convolutionalneural network model.
 10. The system according to claim 7, wherein theimage data is patient pose image data.
 11. The system according to claim7, wherein the hardware processor is further configured to output via auser interface a ranking of confidence scores and enable a selection ofa pose associated with a confidence score.
 12. The system according toclaim 11, wherein the hardware processor is further configured tocompare the selected pose to a required pose of the pre-definedprotocol; and confirm the selected pose as corresponding to the pose ofthe pre-defined protocol and begin the scan; or provide pose adjustmentinstructions if the selected pose is not confirmed.
 13. A computerprogram product comprising a non-transitory computer-readable mediumhaving machine-readable instructions stored thereon, which when executedby a computer causes the computer to: obtain image data from an imagingsensor; classifying the image data, using a first machine learning modelas a patient pose based on one or more pre-defined protocols for patientpositioning; providing a confidence score based on the classification ofthe image data; and if the confidence score is less than apre-determined value, re-classifying the image data using a secondmachine learning model; or if the confidence score is greater than apre-determined value, identify the image data as corresponding to apatient pose based on the one or more pre-defined protocols for patientpositioning.
 14. The computer program product according to claim 13,further comprising execution of the machine-readable instructions by thecomputer to automatically begin a scan according to the one or morepre-defined scanning protocol when the correct patient pose is selected.